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The general problem of equalization for data transmission where 
one of the two data sources produces continuous amplitude data 
samples is studied. There are various ways to configure a modem for 
such a transmission scheme, and we describe how a standard quad- 
rature amplitude modulation structure can be modified to operate in 
this mode. This solution can be specialized to include various linear 
modulation schemes, such as single sideband and vestigial sideband. 
Theoretical analysis shows that adaptive equalization and adaptive 
phase tracking can be achieved with similar quality as in the familiar 
digital-only modem. We provide extensive computer simulation re- 
sults which confirm the validity of our theory. 

I. INTRODUCTION 

Recently, considerable interest arose in finding ways to transmit and 
receive digital and analog data simultaneously over the 2-wire voice- 
band telephone channel. We investigate a system using quadrature 
amplitude modulation (qam) where digital and analog data modulate 
the two quadrature channels. The same scheme was independently 
proposed in Ref. 1. The effects of various channel impairments, as well 
as an imbalance of signal powers between the digital and analog 
signals, on the error probability of the system has been studied in 
Ref. 2. 

In this paper, we analyze an adaptive equalizer for this type of 
hybrid modulation, using a cross-coupled transversal filter as described 
in detail by Falconer in Refs. 3 and 4. The difference here is that one 
of the two quadrature channels transmits analog, i.e. continuous am- 
plitude, data. If the communication channel is time invariant, it is 
possible to train the equalizer with digital data on both quadrature 
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channels and freeze the equalizer taps after convergence. Analog data 
can then be sent. However, possible changes in channel characteristics 
warrant the use of an adaptive equalizer to update the taps continually. 
Since references for the analog data are not available, especially when 
the receiver is in a decision-directed mode, the update algorithms 
reported in Refs. 3, 4 are not applicable in this case. We propose a 
modification which only minimizes the mean-square error (mse) in the 
digital data path. Thus, only the error signal from this path is required 
for adaption. This analysis is similar to that in Refs. 5 and 6. 

Results from computer simulations are given to verify our analytical 
results. We observed that because the analog data does not aid in the 
equalization, it actually acts as an interferer. As such, it would seem 
advantageous to reduce the analog signal power, thus, unbalancing the 
system. On the other hand, this would degrade the analog s/n. There- 
fore, there is a trade off in allocating different power levels to the two 
channels depending on which signal is more important. 

In the scheme described here, we assume that data symbols are sent 
every T seconds. Thus, the analog signal has to be limited in bandwidth 
to 1/(2 T) in order to avoid aliasing. Alternatively, the two quadrature 
channels can be used to transmit primarily analog data with occasional 
digital data for purposes of equalizer updating. Then we could have 
one high-rate analog channel and a low data rate digital channel. 

II. MATHEMATICAL MODEL 

The general qam transmission scheme of Fig. 1 is considered. Two 
data sequences, {a„} and {b n }, are applied to the in-phase and quad- 
rature inputs of the cross-coupled transmission filters with impulse 
responses g p (t) and g q (t). Their output signals are modulated by sine 
and cosine waves of carrier frequency wo to form the passband signal 
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Fig. 1 — Model of transmitter and channel. 
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s(t) = Re lD n G(t-nT)exp(jioot)\ (1) 

where T is the symbol interval, 

D n = a n + jb n (2) 

and 

G(t)=g p (t) + jg q (t). (3) 

In the above equations, {D n } is the complex sequence of data symbols 
and G(t) is the complex impulse response of the transmission filter. 
These parameters can be specialized to represent any linear modula- 
tion scheme, 7 e.g., amplitude/phase modulation, single sideband (ssb), 
vestigial sideband (vsb), and qam. 

Throughout this paper it will be assumed that at least one data 
sequence, for example, {a„} is digital. In particular, we will report 
results for a system with G(t) real and where {a„} and {&„} are digital 
and analog data sequences, respectively. These sequences are assumed 
to have zero-mean and the following correlation properties: 

E {a n a m } = P a S nm (4a) 

EibnbJ^PvSnm (4b) 

E{a n b m )=0 (4c) 
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After passing through a noisy, dispersive, and phase-jittered channel, 
the signal at the input to the receiver can be expressed as 

q(t) = Ref expO'M + M)]} I D k U(t - kT)J + 7i(t), 

where U(t) is the combined complex impulse response of the trans- 
mitting filter and the baseband component of the pass-band channel 
h c (t ) with respect to the carrier frequency wo. The noise process i\(t) is 
independent of the data sequences, while the phase shift 0(t) caused 
by the channel is assumed to vary much more slowly than the channel- 
impulse response, h c (t), and is typically about 10 degrees peak-to-peak. 
It is mutually independent of the additive noise process, as well as of 
the data symbols. 

The general qam receiver is shown in Fig. 2. The received signal q(t) 
is bandpass-filtered by the phase splitter pair with impulse response 
F(t) = fit) + jf(t) where f(t) denotes the Hilbert transform of f(t). 
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Fig. 2 — Receiver structure. 

The pair of outputs of the phase splitter at time AT is written as the 
complex signal sample 



X k = x k + jxk 

= exp[j(uokT + d k )] £ DiHk-i + N k , 



(5) 



where {H k = H(t + kT)) are the samples of the overall complex 
baseband equivalent impulse response, and {N k } are the complex 
samples of the nltered-noise process. The latter process is uncorrelated 
with the signal and has an autocorrelation Rnn- Thus, we have 



E{N n N^}=R NN [(n-m)T] 
E{N n N m ) =0 
E{a n N m ) =0 

E{bnN m }=0 



(6a) 
(6b) 
(6c) 
(6d) 



for all integers n and m. 

The complex signal sequence {X k } is passed through a cross-coupled 
passband equalizer with 2M + 1 complex taps, the output of which at 
time nT is given by 



Qn = £ CkXn-k 
k—M 

= £*2£n, 

where we write the complex tap as 

C k = c k + jc k , c k and c k real 
and define the vectors 

Cf = [C-m, • • • , C M ] 

2Ln = [X n +M, " * * i X n -M J. 



(7) 



(8) 
(9) 
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We use the * to denote conjugation for scalars and conjugate trans- 
position for vectors and matrices. The symbol t denotes transposition. 
The signal Q n is demodulated to baseband by multiplication with 
exp(—j0n — jmnT), where n is the estimated phase offset (or jitter) 
at time nT. The resulting signal Y n can then be written as 

Y n = Q n exp(-yft, - jaonT) (10a) 

- y n + jy n . (10b) 

The demodulated outputs y n and y n are estimates of the transmitted 
data samples. In the following section, an optimum equalizer tap vector 
is derived which minimizes the mse of an appropriate cost function. 

III. OPTIMIZATION OF EQUALIZER COEFFICIENTS 

3. 1 Analysis of the minimum mse criterion for the in-phase branch 

In Refs. 3 and 4 an optimum equalizer that minimizes an mse 
criterion was derived. The mse was defined as 

E[\ Y n - D n \ 2 ] = E[Re 2 (Y n - D n ) + Im 2 ( Y n -D n )] 

= E[(y n - a n ) 2 ] + E[(y n - 6») 2 ], (11) 

which is the sum of the mse's in both branches of the equalizer output. 
It was found that the optimum equalizer coefficients can be calculated 
adaptively provided the complex output error Y„ — D„ is available to 
the receiver. While this is the case for a transmission system with 
digital data in both branches where references can be estimated easily, 
it is not for the system considered here. In this application only one 
reference sequence is assumed to be available. Consequently, only the 
error signal in this branch can be made available for updating purposes. 
In the following discussion, we define an optimum tap vector which 
minimizes the mse in that branch where a reference signal is available 
or can be estimated easily. The resulting tap vector will be compared 
with the result for the case where both references are available. 
Assuming we have a reference for {a„}, we define the mse in that 
branch as the cost function to be minimized 

d - E[(y« - a n ) 2 ] (12) 

with 

y n = Re[ C *£, exp(- j9 m - /WiT)] . (13) 

It is convenient to express y n in vector-matrix notation as follows: 

y n = C'7W„)X„, (14) 

where we partitioned the complex vectors C of the passband equalizer 
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coefficients and X* exp(-y'0„ - juonT) of the ideally demodulated 
received signal to get real vectors C and X„ with twice the original 
dimension 

C' = [Re(C')|Im(C')] (15) 

X^ = {Re[X e n exp(-j0„ - jaonT)] 

= | Im[X'n exp(-j0 n - 7W1T)]} (16) 

M n = 6 n - L (17) 

In eq. (14), T(A0) is a transformation matrix expressing the effect of 
the demodulating phase error A0„ and is defined as 



T(a) = 



cos a # 





—sin a 








' cos a 





* —sin a 
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*sin a 





*cos a 



(18) 



Note that this matrix is orthogonal; that is, 

T(a) X T'(a) = T'(a) X T(a) = I. (18a) 

Furthermore, 

T(-a) = T{a) (18b) 

and 

T(a + iS) = T(a) X T(fi) = T(0) X T(a). (18c) 

In order to get the mse eq. (12) as an explicit function of the coefficient 
vector C, we introduce the autocorrelation matrix A of the demodu- 
lated received signal 

A = E{X„XJ,} (19) 

and the cross-correlation vector V between the demodulated received 
signal and the reference 

Y-J(U). (20) 

With eqs. (14), (19), and (20), we can express eq. (12) as follows: 

d = C'T(M n )AT(-Mn)C - 2C'T(A0 n )V + E(al). (21) 

Setting the partial derivatives with respect to C to zero yields the 
vector equation for the optimum tap vector C op t 



AT(-A0 n )C O pt = V. 



(22) 
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The definition of A in eq. (19) ensures that it is positive definite. 
Consequently, the equation has a unique solution. 

Copt = T(A^„)A- 1 V. (23) 

Inserting eq. (23) into eq. (21) yields for the minimum mse 

eopt = E (al) - Copt T(A0 n ) V = E (a 2 n ) - VA^V. (24) 

In Appendix A, we show that, for stationary data sequences {a„} and 
{bn} uncorrelated with the noise, the autocorrelation matrix A and the 
cross correlation vector V are independent of the time instant n. It is 
important to note that the minimum mse is independent of the 
constant demodulation phase error A0„. This is a consequence of eq. 
(23) and indicates that even by minimizing the mse in one of the two 
equalizer outputs, the optimum coefficient vectors can take care of any 
phase error, in the same manner as in a cross-coupled equalizer which 
minimizes the total mse at the output. 

These facts have been reported in Refs. 5 and 6 for vsb- and ssb- 
modulated pulse amplitude modulation signals. Our analysis shows, 
however, that this holds in general for stationary sequences {a„) and 
{bn}. Thus, the independence of the minimum mse on the demodula- 
tion phase is a property of the cross-coupled equalizer which is not 
adversely affected by the particular selection of the cost function nor 
by the nature of the sequence {b n ). 

When the power of the two data sequences is balanced, i.e., P a — Pb, 
it can be seen from eq. (80) that the resulting equation for the optimum 
tap vector coincides exactly with the equation resulting from minimiz- 
ing the total mse in both branches. In this case, both methods will give 
the same optimal coefficient vector and the same total mse. 

In all our analysis we have assumed uncorrelated data as described 
by eqs. (4a) through (4c). If, instead of eq. (4b), we have 

E{b n b m ) = R b (n-m), 

then the expressions for Ai(&, /) and A 2 (k, I) in Appendix A would be 
more complicated but they would remain stationary matrices. Then, 
assuming correlated data, eq. (24) is in general still valid, except that 
V and A are more complicated than the expressions derived in Appen- 
dix A. 

Although we have based our analysis on a symbol spaced equalizer, 
we can also handle fractional spacing and derive similar results. As an 
example, we can view the T/2 equalizer as two parallel symbol-spaced 
equalizers, where the first of the two data samples during each baud is 
processed by one of the equalizers while the second is processed by the 
other equalizer. Let us denote the two equalizer tap vectors as.£ i and 
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Q_2 and the input vectors as X„ and Xn+m- Then we can define 

C = [Re(Ci) | Im(£i) | Re(Cf ) | Im(C£)] 

X^ = {Re[X' n exp(-./0„ - juonT)] | Im[2££ exp(->tf„ - 7W1T)] 
I Re[2£n+i/2 exp(-j0 n - jwonT)] 
I Im[X^+i/2 exp(-jO„ - jwonT)]} 



T(a) = 



~ Ti(a) 








Trial 



where Ti(a) is given in eq. (18). With these definitions, all the previous 
results for C op t and d pt in eqs. (23) and (24) follow. 



3.2 Mean square error In the quadrature branch 

We now analyzed the mse in the second branch of the equalizer. 
The output of this branch is 

y n = lm[C*X n exp(-jOn - juonT)]. (25) 

Since 



Im{£}=Re|£exp(-./|)l 



(26) 



we are able to express y„ in terms of C and X defined in eqs. (15) and 
(16) using the transformation properties in eq. (18) 



y n = C t T[M n --)X n . 



(27) 



Therefore, the mse in the second branch can be expressed in vector- 
matrix notation as follows: 



el = E[(y n - b n ) 2 ] 



- CX ( A0„ - |J AT (-A0„ + ^ ) C 



-2CT ( A0 - -J 



E(X n bn)+E(b 2 n ). 



(28) 



(29) 



Using eq. (18) and an approach similar to eq. (66) through eq. (70) in 
Appendix A, we show that 



(-D 



E{X n b n }=V(P b /P a ). 



(30) 
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On substituting eq. (30) into eq. (29) we obtain 
H = CT (a0„ - |^ AT (-Ad n + |J C 

-2C'r(A0 n )V^ + P6. (31) 
For the balanced power case, i.e., P a = Pb, Appendix A shows that 

T B) Ar (i) =A ' (32) 

and it follows that 

6 2 = C'T(Mn)AT(-M n )C - 2C'T(M n )V + P a . (33) 

This is exactly the same expression as for e 2 in eq. (21). Consequently, 

P = e 2 . (34) 

Thus, we conclude that the mse's in both branches are equal. In this 
special case, minimizing the mse in one branch also minimizes the mse 
in the other branch. 

3.3 Analysis of an infinitely long equalizer 

While the formal solution eq. (24) for the minimum mse already 
shows the independence of a constant-phase error, it does not reveal 
anything about the influence of channel parameters (amplitude, or 
phase distortion) or of the sampling instant. To obtain further insight 
into this dependence, we analyze an infinite length equalizer. 

We show in Appendix B that the resulting minimum mse for an 
infinite tap equalizer can be written as 

T p a r*/T P b [Z(u>) + Z(-co)] + 1 , , _, 

eopt = -r— AJ3 p „. .„. : du, (35) 

2w J _ /T 4PaPbZ{io)Z i-oi) 

+ (Pa + Pb)[Z(u)+Z(-U))] + l 



where 

2 



Z(w)- 



HJiu) 



Neo(w) 



(36) 



In eq. (36) Heqito) is the Fourier transform of the sampled impulse 
response H (t + kT), where r indicates the sampling instant. It is 
related to the transfer function if (co) as follows: 

a,(«) = e -^ X H ( a + k ¥) exp (»* i) • (37) 
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and | iVeq(co) | 2 is the baseband component of the noise power spectrum 
|iW«)| a - S ^iviv(AT) exp[-y(to + «o)*T]. (38) 

A— 00 

The formula given in eq. (35) can be evaluated for all the different 
modulation schemes which can be modeled by a linear transmission 
system. The only frequency-dependent term appearing in eq. (35) is 
Z(u), the s/n of the sampled received signal. According to eq. (37) this 
will, in general, depend on the sampling instant, t, and the phase 
characteristic of the overall channel transfer function, H(u). If the 
sampling theorem is satisfied, i.e., if H(u ) = for all w outside the 
interval [«i, «i + 2ir/T], where wi is arbitrary, combining eqs. (37) and 
(36) shows that the minimum mse is only dependent on the amplitude 
characteristic of the channel transfer function and of the noise power 
density spectrum. A qam transmission system with no excess band- 
width is an example; a system transmitting only one data sequence 
and using vsb -modulation with less than 100 percent excess bandwidth 
is another, more realistic example. In case of balanced power in the 
transmitted data sequences, i.e., P a = Pb or for Z(w) symmetric around 
«i, i.e., Z(u + wi) = Z(— (o + wi), the mse is given by, 

*,„»r *_ ( 39) 



_.. J ^ /T 2P a Z(u) + l 

In eq. (100) of Appendix B we show that the partial derivative of €o pt 
with respect to Pb is nonnegative. Therefore, an increase in the analog 
signal power Pb causes an increase in el pl . So the analog signal acts as 
an interferer to the digital signal. 



IV. ANALYSIS OF THE GRADIENT ALGORITHM FOR JOINT 
EQUALIZATION AND PHASE TRACKING 

In an adaptive receiver, the equalizer is assumed to know the 
reference data for startup and to operate in a decision-directed mode 
when random data is being sent. In either case, the tap weight vector 
is being updated continuously. Similarly, the phase offset, or jitter, 
would be continuously tracked in order to remain in synchronism. As 
Falconer did, in Refs. 3 and 4, we assume gradient algorithms are used 
in these up da tings as follows (C is now time- varying), 

C„ +1 = C„-|vCd (40a) 

2M n 
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where VCel denotes the gradient of e„ with respect to C. 
It can be shown from eq. (21) that 

VCe* = 2T(Aft 1 )[AT'(Aft l )C Jl - V] (41a) 

and 

4- el = 2C ^ (Mn)[Ar(M H )Cn - V]. (41b) 

dd n ddn 

Using the definitions of A in eq. (19) and V in eq. (20) and substituting 
eq. (14) we can recast eqs. (41a) and (41b) into 

VC n el = 2E[T(M n )X n ( y n - o„)] (42a) 



B6n 



C„ 1 X„(;y„ - a n ) 



(42b) 



Finally, we use another property of the transformation matrix T(a), 
namely, 



ar(a+ ^ = T|a + /5 + ; 



I> 



da 
together with eq. (27) to obtain 

?£ = 2E[y n (yn-an)]. (42c) 

Equations (42a) and (42c) can be used to update the equalizer accord- 
ing to eq. (40). Note that all the signals required to update the equalizer 
are readily available at the receiver. Also note that y„ — a n = e n is the 
error in the branch where a reference is available and y n is the output 
of the other branch. 

By taking expected values as shown in eqs. (42a and b), we obtain 
the estimated gradient algorithm. In practice, a stochastic gradient 
approach is used to avoid the long delay involved in estimating the 
averages. The update equations are obtained by omitting the expec- 
tations in eq. (42) and making small corrections in the direction of the 
instantaneous values instead. Hereafter, we shall discuss only the 
stochastic gradient method. 

Inserting eq. (42a) into eq. (40a) yields the update equation for the 
coefficient vector 

C„ + , = C„ - fi'(y m - a n )T(M n )X n . (43) 

The corresponding equation for the update of the phase jitter corrector 



is 

a' „ 
a 



#n+l - 8n — -^yniyn — On). (44) 
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Division by a„ has been included in eq. (44) to give the corrections a 
smaller weight, if the nominal data value has a larger absolute value. 
Note that because of the special nature of T(A0„) defined in eq. (18), 
the matrix multiplication in eq. (43) requires only 2(2M + 1) multipli- 
cation. Equation (43) can be reexpressed in terms of the complex 
equalizer C_ n and input X n as 

C n+ i = C n - P'KnUyn - a„) expUnuoT + jO n )]*. (45) 

The equalizer coefficient and phase adjustment algorithms repre- 
sented by eqs. (44) and (45) are similar to the ones published in Ref. 
3. The main difference is that here, only the error in the digital data 
path appears in the adjustment algorithms. 

An important parameter in the evaluation of the dynamic behavior 
of the control loop is the rate of convergence (roc). For the case of 
updating the equalizer only, the roc can be analyzed using, for exam- 
ple, Ungerboeck's method, 8 since the stochastic recurrence equations 
for the excess mse can be cast in the form analyzed in Ref. 8. Combining 
eq. (43) with eq. (14), it is seen that for the stochastic gradient case 

T(-A^„)C„ + i = [/ - P'X n X n ]T(-A8 n )Cn + P'a n X n . (46) 

From eqs. (21) and (24) the excess mse is given by 

d - el pt = (C„ - Co Pt )'T(A^)AT(-A^)(C„ - C op t). (47) 

Therefore, 

eJUi - e'pt = DLAD„, (48) 

with 

D„ = (/ - P'X n X n )T(-Mn)(C n - Copt) 

- y8'X„[X^T(-A^)C op t - an]. (49) 

The convergence of the excess mse eq. (48) is analyzed in Ref. 8. There 
it is shown further that for stability the control loop constant has to 
satisfy 

°^idkr (50) 



with 



«-iokr (51) 



The control loop constant, ySo, is chosen to give the fastest initial 
convergence. 
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In steady-state the excess mse is given by 



,, 2 j8'^(X'X)6it 



e " 6opt "2-B^(X t X)' * ' 

From eq. (52) it can be seen that the excess mse can be reduced to an 
arbitrarily small value by selecting /?' small enough. 

It is interesting to note that eq. (50) specifies a stability region for 
/?' equal to that for an equalizer using error signals from both branches 
[£(X'X) = E (£*&)]. From eq. (52) it follows that for a particular 0' 
the ratio of the excess mse to Co P t is the same as for an equalizer using 
error signals from both branches. 

We have not been able to analyze the roc for the joint operation of 
equalizer and phase jitter loops. In contrast to the equalizer update 
equation, the transformation matrix T(A0„) in the joint case is involved 
in a nonlinear manner. We, therefore, resort to computer simulation of 
the loop behavior. These results are reported in the next section. 

V. SIMULATION RESULTS 

Here, we present simulation results for the hybrid modulation 
scheme described earlier. Pseudorandom digital data selected from the 
{±1, ±3} alphabet is used to modulate the in-phase channel, while a 
set of pseudorandom numbers with a Gaussian distribution N(0, Pb) 
modulates the quadrature channel, where Pb is the analog signal 
power. Additive Gaussian noise at —30 dB below the average signal 
level is introduced in the channel and, whenever desired, phase jitter 
is introduced. The latter process is modeled by a 60-Hz sinusoid of 10 
degrees peak-to-peak. The channels used in the simulation are (i) a 
good channel with a flat amplitude frequency response within most of 
the frequency band of interest and small delay distortion, except near 
the lower band edge, and (ii) a channel just violating the requirements 
for basic conditioning with both moderate amplitude and delay distor- 
tion. These characteristics are shown in Fig. 3. 

Although it is not intended to start up an equalizer with a reference 
signal only in one branch, we report results of such simulations. This 
gives good insight into the dynamic behavior of the adaptive equalizer 
update loop and facilitates the comparison with a conventional pass- 
band equalizer. 

The first run described is for balanced power (P a = Pb = 5), where 
the receiver has a 64-tap T/2 complex equalizer and the error signal is 
derived from the in-phase digital channel alone. The basic channel 
described above is used. Figure 4 is a sample simulation run displaying 
the s/n as a function of time where s/n is defined as the ratio of the 
digital signal power to the digital mse. The latter is taken to be a 
weighted sum of past and present instantaneous squared errors. The 
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Fig. 3 — Channel frequency responses. 

two curves are for two different timing phases in the receiver. As 
shown, the equalizer converges in about 2000 iterations for a step size 
of P = 0.0005. The step size giving the fastest convergence, according 
to eq. (53), would be fio = 0.0015. A lower value is used in order to re- 
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Fig. 4 — Convergence of hybrid equalizer. 
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duce the mse in steady state. The receiver's digital and analog outputs 
after equalization and demodulation to baseband are displayed in Fig. 
5 as a scatter plot. The vertical and horizontal axes represent analog 
and digital values respectively and, ideally, the data points would be 
on vertical lines passing through the X-axis at ±1, ±3. Owing to 
channel noise, there is both lateral and vertical displacement from the 
true values. The decision thresholds for the digital data are the vertical 
lines with abscissae and ±2. In this example, and all the others to be 
described, the input s/n is about 30 dB so that the equalizer has done 
a reasonable job in removing intersymbol interference (isi) caused by 
the basic channel. 

We next exhibit the results for the regular qam in Figs. 6 and 7, with 
digital data on both branches and using exactly the same channel and 
receiver parameters as before. The ideal constellation in the scatter 
plot would be 16 points with coordinates ±1, ±3 (in the absence of isi 
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Fig. 5 — Hybrid receiver output constellation. 
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10 20 30 40 

ITERATIONS IN HUNDREDS 

Fig. 6 — Convergence of regular QAM equalizer. 



and channel noise). As seen in Fig. 6, the convergence is faster than for 
the hybrid modulation since here we used a complex error function, 
leading to a tap weight vector C„ that fluctuates less than when using 
a one-sided error signal only. 

We mentioned in Section 3.1 that when the powers in the digital 
and analog branches are balanced, the optimum equalizer and mse are 
the same as that of a regular qam receiver. Indeed, we see by comparing 
Figs. 4 and 6 that the digital mse's reach the same levels. It was also 
observed that the significant tap weights for both receiver timing 
phases differ by less than 5 percent. 

Now we exhibit the effect on the mse of unbalancing the analog 
signal power from Pb = 5 to some other value. The simulated receiver 
has an agc that scales the received signal to an average power of "ten" 
and, hence, the equalizer tap adjustment step size is kept the same in 
this series of runs. The analog mse is obtained by averaging all the 
past instantaneous mse's over time. Instead of presenting a series of 
curves, we summarize the results in Table I. The digital signal power 
is fixed at "five," but the analog power is varied from "one" to "nine." 
The output mse normalized to the power of the corresponding signal 
is presented in Table I for the good channel. Thus, we see that, as the 
analog power is increased, the digital mse increases, confirming the 
analytical results in Section 3.3 for the infinite length equalizer. It was 
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Fig. 7 — Regular qam receiver output constellation. 

also shown in Ref . 2 that the error probability upper bound is increased 
when a power imbalance exists in favor of the analog signal. As 
expected, we also see that the normalized analog mse is smaller with 
a larger analog signal power. We also note that the digital and analog 

Table I — Comparison of analog 

and digital mse for different P b 

(good channel) 



Normalized mse 


Power Pb 


Digital 


Analog 


1 

3 
5 
7 
9 


0.0007 
0.0010 
0.0013 
0.0016 
0.0018 


0.0040 
0.0017 
0.0013 
0.0011 
0.0009 
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mse's are almost the same for balanced power, as theoretically pre- 
dicted in Section 3.2. 

Simulations were also performed to study the hybrid equalizer 
performance in the presence of sinusoidal phase jitter of 60 Hz, where 
the phase tracker modeled by eq. (44) is used to estimate the jitter 
process. Figure 8 shows a sample run where, after allowing the equalizer 
to reach steady-state, 60-Hz jitter with 10 degrees peak-to-peak am- 
plitude is introduced causing a degradation in performance. The plot 
in Fig. 9 shows the same run, except that the phase tracker is turned 
on shortly after the phase jitter is introduced. The lower the jitter 
frequency and amplitude, the more effective we can expect the phase 
tracker to be. 4 

VI. CONCLUSION 

A data transmission system capable of transmitting and receiving 
analog and digital data simultaneously has been studied in detail. We 
found that it is possible to perform adaptive equalization of the channel 
even when only one of the two quadrature channels is carrying digital 
data. Moreover, the minimum mse and tap-weight vector are un- 
changed from that of the regular qam as long as the analog and digital 
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Fig. 8— Hybrid receiver performance in presence of phase jitter (no carrier recovery). 
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Fig. 9 — Hybrid receiver performance in presence of phase jitter (carrier recovery, 
a - 0.5). 

signal powers are equal. However, start-up with simultaneous analog 
and digital data is slower by approximately a factor of two compared 
to the case of a conventional qam system. An efficient start-up proce- 
dure might be to train the receiver with digital data on both channels 
and then switch one channel to analog data upon convergence of the 
equalizer, since in the case of balanced power, the equalizer taps are 
the same. We also found that the scheme can tolerate moderate 
amounts of phase jitter. 



APPENDIX A 
Analysis of A and V 
We define, 

K P .n = Re[X n exp(-jdn - junT ')] 

According to eq. (16), it follows that 

X n = \JLpsi\2Lqji)- 



(53a) 
(53b) 

(54) 
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Inserting eq. (54) into the definition eq. (19) of A yields 



A = E 



-A p.n -A p t n 



This can be expressed as 

, 1 /Re(A a + A 2 ) 
A = - 



JL.p/iJLqji 



Xq,nXq,nj 



-Im(Ai - A 2 ) 



Re(A! - As) 



(55) 



(56) 



2 VIm(Ai + A 2 ) 

where 

A, = E[(X P ^ +jlL,*)ULp.n -jBWl = E(Z n Xl) (57) 

A 2 - E[(X P * +jX g ^)(Xp^ + jX ja , n ) t ] 

= E[XnX' n exp(-2j8 - 2j<x>nT)l (58) 

We note that A 2 is symmetric and Ai is Hermitian. For uncorrected 
data and noise sequences the k, Zth entry in the matrix A 2 is computed 
with eqs. (5), (6), (9), and (58) as follows 

A 2 (k, l) = E[X n -*Xn-iexp{-2j6 n - 2jioonT)] 

= X X E(D v D l )H n -k-pHn-l- l i 

exp[j(O n - k + 6 n -, - 20 n ) -jao(k + l)T] 

+ E[Nv- k Nn-iexp(-2jO n - 2JaonT)]. (59) 

From eqs. (2) and (4) we get 

E(DJ)J = (P a - Pb)8». (60) 

With the assumption that the phase jitter is quasi-stationary over the 
equalizer length, i.e. n -k - On for all k G [-M, M], we obtain 

A 2 (k, l) = (P a - Pt) I HJI^k-iexvi-juoik + l)T]. (61) 

Note that A 2 (A>, Z) is zero, if the powers of the sequences {a„} and {&„} 
are equal, i.e., P a = Pb. 
Similarly the k, Ith entry in Ai is 

Ai(A, l) = E(Xn-kX* n -l) 

= J £ E(DJ);)Hn-k- l Hn'-l- ll 

- M 

X exp[j(S n -k - On-l) + j<*od - k)T] 
+ E[N n -kK-il (62) 

with 

E(DJ);) = (P a + P b )8 v , (63) 
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E(N,Nt) = R NN [(H ~ p)T], (64) 

and the assumption of quasi-stationary phase jitter, this finally yields 

Ai(*, l)-(Pa + Pt) I HJlU-i 

r 

x exp[juo(l - k)T\ + R NN [(k - l)T]. (65) 
With the definition of V in eqs. (20) and (54), we find 

V = Ei?: p ' nan \ 



or 



with 



and 



\KqsianJ 



V, = E[Y.p»On] 



(66) 



(67) 



(68a) 



(68b) 



The kt\\ entry in Yi is calculated by inserting eqs. (5), (9), and (53a) 
into (68b) 

Vi(k) = E{anRe[X n -kexp(-j6 n -juonT)]} 

m E{anRe[exp(jd n -k - j0 n - jwokT) J D t H n -k-i]}. (69) 

Again, under the assumption of quasi-stationary phase jitter and with 
eqs. (2) and (4), we have 



Vi(k) = PaRe[/f-*exp(-/wo*T)]. 
Following the same lines, one obtains for V 2 (k), 

V 2 (k) = P a hn[H- k exp(-jcookT)l 



(70) 



(71) 



A transformed version of A is needed for the analysis of the mse in the 
second branch. According to eq. (18), the transformation matrix is 



(72) 



n-ji- 



r o. o 

" 


1. 0" 

'1 


-1. 
. -1 


0. 
'0. 
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Inserting this into eq. (56) yields 



"-iMi)-i 



Re(A, - A 2 ) 



Im(Ai - A 2 ) 



-Im(A 



Re(A 



li + A 2 ) l 
i + A 2 ) J' 



By comparing eq. (73) with eq. (56), it is found that 



with 



rl-f)AWfl- A + A- 



A' = 



-ReA 2 


— ImA 2 


— ImA 2 


ReA 2 



(73) 



(74) 



(75) 



In eq. 75, A' is symmetric but not positive definite. Then, referring to 
eq. (61), we see that for the balanced power case, i.e., P a = Pb, 

A' = and T{-^\at(^\=A. 



APPENDIX B 

Opt/mum Tap Weight and mse of the Infinitely Long Equalizer 

Define 

C' opt = (Ci | C 2 ) (76) 

and use eqs. (56) and (67). Then, the equation for the optimum 
coefficient vector eq. (22) can be expressed as follows 



Re(Ai + A 2 )Cj - Im(Ai - A 2 )C 2 = 2£, 
Im(Ai + A 2 )Ci + Re(Ai - A 2 )C 2 = 2V 2 , 



(77) 



where it is assumed that the demodulation phase error A0„ equals zero. 
(It has been stated that the minimum mmse is independent of A0„, at 
least for a constant phase offset. Therefore, no loss of generality results 
because of this simplification). 
Let 



C=(C 1 +jC 2 ) 



and 



(78) 



(79) 



V=(¥i+jV 2 ). 
This allows us to write eq. (77) in complex notation 

AiC + A 2 C* = 2V. (80) 

Equation (80) can be expressed in the components of the vector V and 
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matrices Ai and A 2 



I [Ai(*,J)C(Z) + Aa(*,Z)C*(Z)]-2V(A). (81) 

/ — M 

Note from eq. (61) that since A2 is not a Toeplitz matrix, the sum in 
eq. (81) is not a convolution even if the dimension M approaches 
infinity. Define 

C(l) = C(/)expO<ooZT), (82) 

V(D = V(Qexp(jt#>lT), (83) 

Ai(A - Z) = Ai(*. Oexp[-y<oo(/ - k)T], 

= (P a + P b ) l HM^k-i 

+ R NN (k - Qexp[-jMl - k)T], (84) 

A 2 (fc - Z) = A 2 (k, Z)exp[>«o(A - Z)T], 

= (P a - P 6 ) X i?^^- A -/. (85) 

Equations (61) and (65) were used to get eqs. (84) and (85). With these 
definitions, we can transform the problem to baseband. Inserting eqs. 
(82) to (85) into eq. (81) yields 

I [& 1 (k-l)£(l)+&2(k-l)C*(Q] = 2V(k). (86) 

/ — M 

Here the sums will be convolutions if M — > 00. Consequently, the 
equation can be expressed in the domain of Fourier transform as 

Ai(w)£(w) + A 2 (u)£*(-u) = 2V(co). (87a) 

Together with the transform of the conjugate complex of eq. (86), 
which is 

A,*(-«)C*(-w) + A 2 * (-«)£(«) = 2? *(-«), (87b) 

it can be used to find the formal solution 

~ 2P(«)A?(-«)-2P*(-«)Aa<«) ._.. 

C(u) = — — . (88) 

Ai(w)Af (— 10) - A 2 (to)A2*(-<o) 

The Fourier transform of the sampled impulse response H(t + kT), 
where t indicates the sampling instant, is defined as follows 

#«,(«)= i H(t + kT)exp(-ju)kT), (89a) 

K — 00 
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where 

H(t + kT)=^2 | H*a(a)eip(ju)kT)db). (89b) 

w/T 



- t r h 



Assuming the impulse response H(t) has the Fourier transform //(to), 
Poisson's sum formula can be used to get the following relation 
between the spectra of the sampled and the continuous functions 

«*4 = '-^f^ J_ «(« + 2, J) exp(,2*A l). (90) 

Using the spectral density of the filtered noise in baseband 

|iV«,(«)| 2 - S R NN (kT)exp[-j(a + uoHkT)] (91) 

and eqs. (89a) and (89b), the transforms of Ai, A2, and V can be shown 
to be 

Ai(») = (P a + P b ) | H eq (-co) | 2 + | Ar eq (-«) | 2 , (92a) 

As(u) = (Pa - P^lu^N, (92b) 

f («) = PaHmti-u). (92C) 

From eq. (24) and with eqs. (67), (76), and (78) the minimum mse can 
be expressed as 

e 2 pl = Pa - CSptV = Pa - Re(£*Y). (93) 

In eq. 93, C*V can be viewed as the zeroth term of the convolution of 
the sequences (Vn) represented by V and (C*„) represented by C*. 
Multiplying the spectra and transforming back to time domain yields 

•w/T 

(94) 

v. - 
'-w/T 



t r /T i 

Re(£*Z) = — - [C*(«)V(«) + C(-«) V*(-io)]du. 

J-ir/T 



Taking into account the modulation in eqs. (82) and (83), this also can 
be expressed in terms of C(u) and V(co), which are defined above. 



t r /T 1 



Combining eqs. (89a), (89b), (92a), (92b), (93), and (95) finally yields 
the desired expression of the minimum mse 

•w/T 



-w 



.x a j P 6 [Z(«) + Z(-«)] + 1 



^'-^U aawgxw *- (96) 

+ (Pa + Pft) X [Z(W) + Z(w)] + 1 
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where 



Z(co) = 



N eq (o>) 



(97) 



In the balanced power case P a = Pb, we can simplify further to yield 



*r 2 — 
€opt — 



TPa 

2m 



r/T 



-w/T 



+ 



2[2P a Z(a) + 1] 2[2P a Z(-u3) + 1] 



du. (98) 



Since both terms are integrated over one full period this yields 



« 2 — 

copt — 



TPa 

~2tT 



■n/T 



-*IT 



dco 



2P a Z(u) + 1 



(99) 



It can be shown that the derivative of the rninimum mse eq. (96) with 
respect to the power in the second branch is 



d€ pt 
UK 



TPl 

2vr 



r/T 



-WT 



[Z( u) - Z(-oi)fdo) 
I? 



(100) 



where D is the denominator of the integrand in eq. (96). This means 
the derivative is positive unless Z(w) is symmetric in which case the 
former is zero. In general, a decrease in the power of the second branch 
will decrease the minimum mse in the first branch. 
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